Systems and methods for data and model-driven image reconstruction and enhancement

ABSTRACT

Systems and methods are disclosed for image reconstruction and enhancement, using a computer system. One method includes acquiring a plurality of images associated with a target anatomy; determining, using a processor, one or more associations between subdivisions of localized anatomy of the target anatomy identified from the plurality of images, and local image regions identified from the plurality of images; performing an initial image reconstruction based on image acquisition information of the target anatomy; and updating the initial image reconstruction or generating a new image reconstruction based on the image acquisition information and the one or more determined associations.

RELATED APPLICATION(S)

This application is a continuation of pending U.S. application Ser. No.14/835,032, filed Aug. 25, 2015, which is a continuation of U.S.application Ser. No. 14/310,685, filed Jun. 20, 2014, now U.S. Pat. No.9,153,047, which is a continuation of U.S. application Ser. No.14/291,465, filed May 30, 2014, now U.S. Pat. No. 8,917,925, whichclaims priority to U.S. Provisional Application No. 61/972,056 filedMar. 28, 2014, the entire disclosures of which is are herebyincorporated by reference in their entirety.

FIELD OF THE INVENTION

Various embodiments of the present disclosure relate generally tomedical imaging and related methods. More specifically, particularembodiments of the present disclosure relate to systems and methods fordata and model-driven image reconstruction and/or enhancement.

BACKGROUND

Medical imaging and extraction of anatomy from imaging is important, asevidenced by the many means of medical imaging available. Severalimaging techniques involve reconstruction and image enhancement on rawacquired data in order to produce better images. Reconstruction andenhancement may be used to decrease noise in an image, smooth theeffects of incomplete data, and/or optimize imaging. Common forms ofmedical imaging that employ image reconstruction and/or enhancementinclude computed tomography (CT) scans, magnetic resonance imaging (MR),ultrasound, single positron emission computed tomography (SPECT), andpositron emission tomography (PET). One mechanism used to achievehigher-quality reconstruction and enhancement is to use priorinformation about a target reconstructed/enhanced image. Typically, theprior information takes the form of assumptions about image smoothnessor image patches from reference images.

Reference images are often available and used to obtain the priorinformation. Reference images may include at least a portion of a targetanatomy, and portions of reference images may be used to render modelsof anatomy associated with the target anatomy. For example, referenceimages may be idealized images, images of a patient associated with atarget anatomy (e.g., wherein a target anatomy may include an anatomicalpart of the patient), images of the anatomical part of other patients,etc. The images may be collected at various times or conditions, andthey may have various levels of relevance or resemblance to a specifictarget anatomy.

Use of the reference images as image patches may mean that referenceimage use is piecemeal and/or may apply only to regions of an imageidentified as problematic. Evaluation of whether reference images aresuitable for use as image patches may be lacking. In addition, use ofreference images only as image patches may mean that unless portions ofan image are identified as problematic, the image or various portions ofthe image may not have the opportunity to benefit from comparison to areference image.

Accordingly, a need exists for systems and methods for reconstructingand enhancing images based on reference images and associated anatomicalmodels.

SUMMARY

According to certain aspects of the present disclosure, systems andmethods are disclosed for image reconstruction and enhancement. Onemethod of medical image reconstruction includes: acquiring a pluralityof images associated with a target anatomy; determining, using aprocessor, one or more associations between subdivisions of localizedanatomy of the target anatomy identified from the plurality of images,and local image regions identified from the plurality of images;performing an initial image reconstruction based on image acquisitioninformation of the target anatomy; and updating the initial imagereconstruction or generating a new image reconstruction based on theimage acquisition information and the one or more determinedassociations.

In accordance with another embodiment, a system for medical imagereconstruction comprises: a data storage device storing instructions forimage reconstruction and enhancement; and a processor configured for:acquiring a plurality of images associated with a target anatomy;determining, using a processor, one or more associations betweensubdivisions of localized anatomy of the target anatomy identified fromthe plurality of images, and local image regions identified from theplurality of images; performing an initial image reconstruction based onimage acquisition information of the target anatomy; and updating theinitial image reconstruction or generating a new image reconstructionbased on the image acquisition information and the one or moredetermined associations.

In accordance with yet another embodiment, a non-transitory computerreadable medium for use on a computer system containingcomputer-executable programming instructions for medical imagereconstruction is provided. The method includes: acquiring a pluralityof images associated with anatomy of a target anatomy; determining,using a processor, one or more associations between subdivisions oflocalized anatomy of the target anatomy identified from the plurality ofimages, and local image regions identified from the plurality of images;performing an initial image reconstruction based on image acquisitioninformation of the target anatomy; and updating the initial imagereconstruction or generating a new image reconstruction based on theimage acquisition information and the one or more determinedassociations.

Additional objects and advantages of the disclosed embodiments will beset forth in part in the description that follows, and in part will beapparent from the description, or may be learned by practice of thedisclosed embodiments. The objects and advantages of the disclosedembodiments will be realized and attained by means of the elements andcombinations particularly pointed out in the appended claims.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the disclosed embodiments, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate various exemplary embodiments andtogether with the description, serve to explain the principles of thedisclosed embodiments.

FIG. 1A is a block diagram of an exemplary system and network for imagereconstruction and/or enhancement, according to an exemplary embodimentof the present disclosure.

FIG. 1B is a block diagram of an exemplary overview of a training phaseand production phase for image reconstruction and/or enhancement,according to an exemplary embodiment of the present disclosure.

FIG. 2A is a block diagram of an exemplary method for a training phaseof image reconstruction and/or enhancement, according to an exemplaryembodiment of the present disclosure.

FIG. 2B is a block diagram of an exemplary method for building a modelof image regions associated with a localized anatomy, for use in atraining phase of reconstruction and/or enhancement of medical images,according to an exemplary embodiment of the present disclosure.

FIG. 2C is a block diagram of an exemplary method of a production phaseof reconstruction of medical images, according to an exemplaryembodiment of the present disclosure.

FIG. 2D is a block diagram of an exemplary method for producing aconverged image reconstruction, for use in a production phase ofreconstructing medical images, according to an exemplary embodiment ofthe present disclosure.

FIG. 2E is a block diagram of an exemplary method a production phase forproducing an enhancement of medical images, according to an exemplaryembodiment of the present disclosure.

FIG. 3A and FIG. 3B are block diagrams of exemplary training methods foriterative reconstruction of images, according to an exemplary embodimentof the present disclosure.

FIG. 4A and FIG. 4B are block diagrams of exemplary methods forproducing reconstructions, according to an exemplary embodiment of thepresent disclosure.

DESCRIPTION OF THE EMBODIMENTS

Reference will now be made in detail to the exemplary embodiments of theinvention, examples of which are illustrated in the accompanyingdrawings. Wherever possible, the same reference numbers will be usedthroughout the drawings to refer to the same or like parts.

As described above, the use of reference images as image patches formedical image reconstruction and/or enhancement may involve using aportion of a reference image to compensate for deficits in a constructedimage. However, the reference images may have little or no impact onother parts of the constructed image. Thus, the present disclosure isdirected to systems and methods for data and model-driven imagereconstruction and enhancement using target anatomy reference images asmore than image patches. In other words, the present disclosure isdirected to improving image reconstruction and/or enhancement byincorporating into image reconstruction and/or enhancement, associationsbetween anatomical subdivisions and image regions available fromreference images.

The present disclosure is directed to a new approach for reconstructionand/or enhancement of a target anatomy image using prior informationabout a target reconstructed/enhanced image, where the informationincludes associations between reference image regions and parts of thetarget anatomy, such as anatomical features extracted from or identifiedin the image regions. In one embodiment, the present disclosure mayinclude both a training phase and a production (and/or usage phase) foruse in a method of image reconstruction, as well as a method ofenhancing images. In one embodiment, the training phase for both imagereconstruction and image enhancement may include developing a set ofknown or knowable associations between anatomy and image renderings. Forexample, in general, the training phase may involve receiving acollection of images, receiving or inputting information of ananatomical part or portion shown in each of the images (e.g., alocalized anatomy for each of the images), and building a model of imageregions associated with respective portions of the localized anatomy. Anoutput from the training phase may include a set of anatomicalsubdivisions associated with image regions.

In general, the production phase for reconstructions may include usingthe set of anatomical subdivisions associated with image regions (fromthe training phase) in conjunction with image acquisition informationfor a particular target anatomy, e.g., a particular patient orindividual, in order to create a more accurate and/or better-informedimage reconstruction. In one embodiment, image reconstruction may bebased on acquired images and/or image acquisition information, and imageenhancement may be based on any image information. The production phasefor image enhancement may then include using the set of anatomicalsubdivisions associated with image regions along with image informationto output an enhanced image.

Referring now to the figures, FIG. 1A depicts a block diagram of anexemplary environment of a system and network for data and model-drivenimage reconstruction and enhancement. Specifically, FIG. 1A depicts aplurality of physicians 102 and third party providers 104, any of whommay be connected to an electronic network 100, such as the Internet,through one or more computers, servers, and/or handheld mobile devices.Physicians 102 and/or third party providers 104 may create or otherwiseobtain images of one or more patients' cardiac, vascular, and/or organsystems. The physicians 102 and/or third party providers 104 may alsoobtain any combination of patient-specific information, such as age,medical history, blood pressure, blood viscosity, etc. Physicians 102and/or third party providers 104 may transmit the cardiac/vascular/organimages and/or patient-specific information to server systems 106 overthe electronic network 100. Server systems 106 may include storagedevices for storing images and data received from physicians 102 and/orthird party providers 104. Server systems 106 may also includeprocessing devices for processing images and data stored in the storagedevices. Alternatively or in addition, the data and model-driven imagereconstruction and enhancement of the present disclosure (or portions ofthe system and methods of the present disclosure) may be performed on alocal processing device (e.g., a laptop), absent an external server ornetwork.

FIG. 1B is a diagram of an overview 110 of an exemplary training phaseand an exemplary production phase for image reconstruction andenhancement, according to an exemplary embodiment of the presentdisclosure. In one embodiment, the systems and methods for imagereconstruction and/or enhancement may include a training phase 111 and aproduction phase 121. In general, the training phase 111 may involvegenerating associations between anatomical subdivisions and imageregions. The production phase 121 may generally then use theassociations to determine image priors for regions within areconstruction or, in the case of an image enhancement, a previouslyprovided image.

In one embodiment, the training phase 111 may begin with receivinginputs of images 113 and known anatomy 115. Images 113 may includeimages from any known medical imaging modality (e.g., CT, MR, SPECT,etc.). Anatomy 115 may be 2-D, 3-D, or other geometric models of humananatomy. In other words, images 113 may include representations ofanatomy 115, and/or anatomy 115 may show or represent geometry of someportion of anatomy rendered in images 113. For example, anatomy 115 mayinclude models of anatomy, expected anatomy, etc. that are shown (orexpected to be shown) in the images 113. Models of common anatomyrendered between images 113 and anatomy 115 and/or a region of interestin both images 113 and anatomy 115 may be referred to as “localizedanatomy” within each of the images 113. In one embodiment, theassociated images 113 and anatomy 115 may be obtained from the sameindividual for whom images are to be reconstructed and/or enhanced in aproduction phase. In some cases, one individual or patient may be thesource of multiple pairs or even all of the pairs of associated images113 and anatomy 115. In some cases, each associated image 113 anatomy115 pair may be obtained from a different individual or patient. Giventhe input of images 113 and anatomy 115, the training phase 111 may theninclude step 117 of creating associations between portions of anatomy115 and regions of images 113. Specifically, as described in more detailbelow, step 117 may include identifying a region or subset of an image113, identifying a region or subset of a paired anatomy 115, andassociating the region or subset of the image 113 with the region orsubset of the anatomy 115. The training phase 111 thus produces output119, which includes a set of associations between portions of anatomy115 and regions of images 113.

Output 119 may be used as an input to an exemplary production phase 121,where reconstruction engine 123 and enhancement engine 125 may determineimage priors based on output 119 for use in producing reconstructedand/or enhanced images of a particular individual or patient. Forexample, reconstruction engine 123 may receive image acquisitioninformation 127 of an area of anatomy for a particular patient. Usingimage acquisition information 127 along with image priors determinedfrom output 119, reconstruction engine 123 may produce reconstruction129. For image enhancements, enhancement engine 125 may receive imageinformation 131. Enhancement engine 125 may then produce imageenhancement 133 based on image information 131 and image enhancementsdetermined from output 119.

FIGS. 2A and 2B depict flowcharts of exemplary embodiments of thetraining phase 111 of FIG. 1B. FIGS. 2C-2E depict flowcharts ofexemplary production phases for image reconstruction and imageenhancement. FIGS. 3A and 3B depict flowcharts of exemplary embodimentsof training phases as applied to cardiac and abdominal images,respectively. FIGS. 4A and 4B depict flowcharts of exemplary embodimentsof production phases for cardiac and abdominal images, respectively, inwhich the training phase from FIG. 3A may provide an input for theproduction phase of FIG. 4A, and the training phase of FIG. 3B may beassociated with the production phase of FIG. 4B.

FIG. 2A is a block diagram of an exemplary training phase for producinga model of image regions associated with anatomy portions for bothreconstruction and enhancement of medical images, according to anexemplary embodiment. In one embodiment, while the procedures forproduction in image reconstruction and production in image enhancementmay differ in some respects, the procedure for a training phase may, insome cases, be the same for both image reconstruction and imageenhancement. A model of image regions relied upon for the reconstructionand enhancement may be generated the same way. In other words, models ofimage regions for image reconstruction and enhancement may both include,or be based on, a set of known or created associations betweenanatomical subdivisions and corresponding image regions. The set ofassociations may represent an understanding of an image region being arepresentation of a portion of an anatomy, and in some embodiments, anunderstanding of the identity of the person having that portion of theanatomy. The training phase may develop a model of relationships betweenimages and anatomy, based on a collection of images. In this way, amodel of image regions developed from the training phase may form abasis of expected image regions in relation to portions of anatomy, thusproviding guidance for image reconstruction and enhancement. FIG. 2Bdepicts an embodiment of certain steps of the method of FIG. 2A,including exemplary detailed steps for building a model of associationsbetween image regions and anatomy, according to one embodiment.

FIG. 2C depicts steps of an exemplary production phase for an imagereconstruction, according to an exemplary embodiment. FIG. 2D depicts anembodiment of certain exemplary steps of the method of FIG. 2C,including certain steps that may be repeated until convergence in orderto produce the image reconstruction output by the method of FIG. 2C.FIG. 2E includes a production phase for an image enhancement. The stepsin FIG. 2C may be similar to those of the method of FIG. 2E, except thatthe steps for FIG. 2E may not necessarily be based on an acquired image.Rather, since FIG. 2E addresses image enhancement, an image may bealready available and need not be acquired and/or created in anindependent step.

As introduced above, FIG. 2A is a block diagram of an exemplary method200 of a training phase for reconstruction or enhancement of medicalimages, according to an exemplary embodiment of the present disclosure.Method 200 may be performed by server systems 106, based on information,images, and data received from physicians 102 and/or third partyproviders 104 over electronic network 100. The method of FIG. 2A mayinclude receiving a collection of images (step 201). The collection ofimages may include or be associated with a target anatomy, for example,an anatomical feature of one or more individuals. A target anatomy maybe any image and/or portion of an image that may undergo analysis and/orbe used for analysis. In one embodiment, the images may be stored,input, and/or received on an electronic storage device.

In one embodiment, method 200 may further include receiving orinputting, for each image of the collection, a localized anatomy modelof anatomy reflected within the image (step 203). For example, thelocalized anatomy may include a portion of an anatomy to be reviewed oranalyzed. For instance, a target anatomy may include a patient's heart,where a localized anatomy may include a localized model of a coronaryartery vessel tree. In one embodiment, the localized anatomy within theimage may be received on an electronic storage device.

Next, step 205 may include building a model of image regions associatedwith portions of the localized anatomy. In one embodiment, the model maybe built using a computational device. Exemplary methods of building themodel are further described in FIG. 2B. Given the model, a set ofassociated anatomical subdivisions and image regions may be produced.Such a set of associated anatomical subdivisions and image regions maybe output to an electronic storage device (step 207).

FIG. 2B is a block diagram of an exemplary method 220 for building amodel of image regions associated with respective/corresponding portionsof localized anatomy in a training phase for reconstruction orenhancement of medical images, according to an exemplary embodiment ofthe present disclosure. In one embodiment, method 220 is one way ofcarrying out step 205 of modeling associations between image regions andportions (e.g., subdivisions) of localized anatomy. Method 220 may beperformed by server systems 106, based on information, images, and datareceived from physicians 102 and/or third party providers 104 overelectronic network 100. In other words, the model of image regions maybe built using a computational device.

In one embodiment, step 221 may include determining a size and/or typeof subdivision for a target anatomy in the images. For example, asubdivision may be a single component encompassing an entire localizedanatomy. Alternately, subdivisions may be very small relative to theimage. Step 221 may include determining a level of granularity in thesize of the subdivisions. In some embodiments, the size of subdivisionsmay be static or dynamic. For example, step 221 may include adjustingsizes of subdivisions in view of image resolution, sensitivity, etc.

In one embodiment, method 220 may further include step 223 ofsubdividing the localized anatomy into one or more subdivisions, foreach image and localized anatomy in the collection (e.g., the collectionof images received at step 201). For example, the subdivisions may beuniform across the entire image and localized anatomy, throughout thecollection. In another example, the subdivisions may vary, depending ona local region of the anatomy.

In one embodiment, step 225 may include associating a local region of animage with the one or more subdivisions of the anatomy. In other words,regions of the images may not be directly identified as being associatedwith a localized anatomy or one or more subdivisions of the localizedanatomy. Step 225 may create associations between the regions of imagesand the one or more subdivisions, such that the local regions of theimages may be recognized as being associated with subdivisions thatcorrespond to the same localized anatomy. In one embodiment, step 227may include an option to determine whether another image is available inthe collection of images (e.g., from step 201). If more images remain inthe collection, the method may continue to subdivide the localizedanatomy in the image (step 223) and associate a local region of an imagewith one or more subdivisions (step 225). If all of the images in thecollection have been through steps 223 and 225, results of thesubdividing and image association may be processed in step 229. In oneembodiment, step 229 may include combining or integrating a set of thelocal regions of the image that are associated with the one or moresubdivisions. In integrating the set, step 229 may build a model ofimage regions associated with respective portions of an anatomy.

FIG. 2C is a block diagram of an exemplary method 240 for producing areconstruction of medical images, according to an exemplary embodimentof the present disclosure. Method 240 may be performed by server systems106, based on information, images, and data received from physicians 102and/or third party providers 104 over electronic network 100. In oneembodiment, method 240 may be based on output from the training phase,for example, method 200 (including method 220).

In one embodiment, method 240 may include step 241 of receiving imageacquisition information, for instance, on an electronic storage device.In one embodiment, step 243 may include performing an initial imagereconstruction based on the acquisition information from step 241. Thereconstruction may be performed using any reconstruction method known inthe art. Step 245 may include receiving a set of associated anatomicalsubdivisions and associated image regions (e.g., from step 207 of themethod 200 of a training phase). The set of associated anatomicalsubdivisions and associated image regions may be received on anelectronic storage device.

Next for step 247, a converged reconstruction may be created using theinitial reconstruction, in conjunction with the set of anatomicalsubdivisions and associated image regions (e.g., from step 245).Exemplary steps for creating the converged reconstruction may be foundat FIG. 2D. Then, method 240 may further include outputting theconverged image reconstruction, for example, to an electronic storagedevice and/or display (step 249).

FIG. 2D is a block diagram of an exemplary method 260 for producing theconverged image reconstruction (e.g., of step 247), according to anexemplary embodiment of the present disclosure. In other words, thesteps of method 260 may be repeated until images converge, thus formingan image reconstruction (e.g., the converged reconstruction). Method 260may be performed by server systems 106, based on information, images,and data received from physicians 102 and/or third party providers 104over electronic network 100.

In general, method 260 of FIG. 2D may include localizing anatomy withinan initial image reconstruction, subdividing the localized anatomy, andperforming image reconstruction using the image acquisition informationand image priors, where the reconstruction is based on expectedassociations between subdivisions and image regions developed from thetraining phase. In one embodiment, step 261 may include localizinganatomy within an image reconstruction, e.g., the initial imagereconstruction from step 243. For example, out of an imagereconstruction, step 261 may include determining an anatomy that is partof the image and pinpointing the anatomy for analysis. Then, step 263may include subdividing the localized anatomy into one or moresubdivisions. In one embodiment, the subdivisions may be uniform, whilein another embodiment, subdivisions may vary across the localizedanatomy. In yet another embodiment, subdivisions of the localizedanatomy for step 263 may differ from subdivisions defined in thetraining phase (e.g., step 223). Step 265 may include determining imagepriors for one or more regions within the image reconstruction, whereinthe image priors may be based on the set of associated anatomicalsubdivisions and image regions from the training phase (e.g., from step207). In one embodiment, the set from step 207 may be the input fromstep 245. In one embodiment, step 267 may then include performing animage reconstruction using acquisition information (e.g., from step 241)and image priors (e.g., from step 265).

From this image reconstruction from step 267, steps 261-267 may thenrepeat until convergence. For example, method 260 may repeat such thatthe reconstruction of step 267 is used as input, wherein anatomy withinthe reconstruction from step 267 is localized (e.g., step 261), thisanatomy is subdivided (e.g., step 263), image priors are found (and/orupdated) from regions within the reconstruction (e.g., step 265), and anew (and/or updated) image reconstruction is produced from theacquisition information and found/updated image priors. In short, method260 may provide one way of producing an image reconstruction from theinputs outlined in method 240. Upon convergence, step 247 may registerthe convergence and determine and/or receive the convergedreconstruction.

As previously stated, method 240 (and method 260) for producing areconstruction may be analogous to a method for enhancing images. Whilethe methods may be similar, deviations between the production phase forimage enhancement versus the production phase of reconstructions areexplained in more detail below.

FIG. 2E is a block diagram of an exemplary method 280 for producing anenhancement of medical images, according to an exemplary embodiment ofthe present disclosure. Method 280 may be performed by server systems106, based on information, images, and data received from physicians 102and/or third party providers 104 over electronic network 100. In oneembodiment, method 280 of producing an enhancement may differ frommethod 240 of producing a reconstruction in that an enhancement is animprovement of an available image. Therefore, in one embodiment, method280 does not include steps of acquiring images or creating an initialimage. Rather, step 281 may start at receiving image information, asopposed to step 241 of receiving image acquisition information. In oneembodiment, step 281 may include receiving image information, forexample, on an electronic storage device. Step 283 may be similar tostep 245 in that a set of associated anatomical subdivisions andassociated image regions may be received, based on a training phase.Again, this set of associated anatomical subdivisions and associatedimage regions may be received from an electronic storage device.

Since method 280 includes an enhancement, an image is already availableand a step of generating an initial image (e.g., step 243) may beunnecessary. In one embodiment, step 285 of performing image enhancementmay include localizing anatomy within the image being enhanced,subdividing the localized anatomy into one or more subdivisions, usingthe set of associated anatomical subdivisions and image regions (e.g.,from step 283) as image priors for one or more regions within the image,and performing image enhancement using image information (e.g., fromstep 281) and the image priors (e.g., from step 283). Then, step 287 mayinclude outputting an enhanced image, for example, to an electronicstorage device and/or display.

FIGS. 3A, 3B, 4A, and 4B are directed to specific embodiments orapplications of the exemplary methods discussed in FIGS. 2A-2E. Forexample, FIG. 3A and FIG. 3B depict exemplary training phase methods foriterative reconstruction of cardiac and abdominal images, respectively,according to various embodiments. FIG. 3A may further provide the basisfor a training phase method for cardiac image enhancement. FIGS. 4A and4B, respectively, include exemplary production phase methods foriterative reconstruction of cardiac and abdominal images. FIG. 4A mayadditionally provide a basis for a production phase method for cardiacimage enhancement. In some embodiments, the output of coronary arterymodels and associated image regions from the training phase of FIG. 3Amay serve as an input for a cardiac image reconstruction productionphase as shown in FIG. 4A. Similarly, surface mesh models and associatedimage regions output from FIG. 3B may be used toward a creating aconverged image reconstruction from the production phase of FIG. 4B.While the embodiments for cardiac and abdominal images are presented asseparate embodiments, the methods applied may be combined intoreconstructions and/or enhancements that simultaneously include variousanatomical parts.

FIG. 3A is a block diagram of an exemplary method 300 for iterativereconstruction of, specifically, cardiac images, according to variousembodiments. For the method 300, cardiac images may include CT imagesand/or MR images. In one embodiment, step 301A may include inputting orreceiving a collection of cardiac CT images, for example, on anelectronic storage device. Iterative reconstruction for producing acardiac CT image may allow for producing a cardiac CT image with a lowerradiation dose by acquiring fewer samples and using prior information toreconstruct a complete CT image. Another embodiment may include step301B of inputting or receiving a collection of cardiac MR images, forexample, on an electronic storage device. For production of cardiac MRimages, partial or parallel reconstruction allows for faster acquisitiontime by acquiring fewer samples and using prior information toreconstruct a complete MR image.

In one embodiment, step 303 may include inputting or receiving, for eachimage, a localized model of a coronary artery vessel tree within thatimage, for example, on the electronic storage device. The coronaryartery vessel tree model may include centerlines of vessels that aresampled at discrete points. Step 305 may include building a model ofimage regions associated with one or more points along one or morecenterlines. For example, for each image in the collection, a geometric(e.g., square or rectangular) region of the image may be associated witheach centerline point in a model. In one case, the geometric region maybe a 5 mm 3-D geometric region. The size of the image region forassociating with a centerline point in a model may be static and/ordynamic, depending, at least, on the images, density of centerlinepoints, processing power, etc. In one embodiment, step 305 of building amodel may be performed by a computational device. Final step 307 mayinclude outputting a set of coronary artery models and associated imageregions, for example, to an electronic storage device.

In one embodiment, the training phase for image enhancement of a cardiacCT image may be similar to the training phase for iterativereconstruction of cardiac CT images. Image enhancement may be a methodfor using prior information to produce cardiac CT images with improvedimage quality and interpretability. One possible distinction may beinputting a collection of good quality cardiac CT images (e.g., on anelectronic storage device), rather than inputting any collection ofcardiac CT images. The training phase for image enhancement may focus onimproving an image using the foundation of good quality cardiac images,whereas iterative reconstruction may provide a set of coronary arterymodels and associated image regions for a specific patient. Remainingsteps for image enhancement of a cardiac CT image may includesimilarities to those for iterative reconstruction, in one exemplaryembodiment. For example, image enhancement may also include inputting,for each image (of the collection of good quality cardiac CT images), alocalized model of a coronary artery vessel tree within that image on anelectronic storage device. The coronary artery vessel tree model mayinclude centerlines of vessels sampled at discrete points. Acomputational device may then be used to build a model of image regionsassociated with the centerlines by, for example, associating a 5 mm 3-Dgeometric (e.g., rectangular) region of an image with each centerlinepoint in a model. Afterwards, a set of coronary artery models andassociated image regions may be output to an electronic storage device.

FIG. 3B is a block diagram of an exemplary method 320 for iterativereconstruction of abdominal CT images, according to one embodiment.Iterative reconstruction may permit production of an abdominal CT imagewith a lower radiation dose, for example, by acquiring fewer samples andusing prior information to reconstruct a complete CT image. In oneembodiment, step 321A may include inputting or receiving a collection ofabdominal CT images, for example, on an electronic storage device.Alternately or in addition, step 321B may include inputting or receivingabdominal MR images, perhaps also on an electronic storage device. Foreach image, step 323 may include inputting a localized model ofabdominal organs (e.g., liver, kidney, spleen, gall bladder, etc.)within that image (e.g., on the electronic storage device). The organmodels may include surface meshes that are sampled at discrete points.In one embodiment, step 325 may include building a model of imageregions associated with the surface mesh points. For example, for eachimage in the collection, step 325 may include associating a geometricregion of the image with each surface mesh point in the model. In onecase, the geometric region may be a 5 mm 3-D rectangular region of themodel. In one embodiment, a computational device may be used to performstep 325. Step 327 may include outputting a set of surface mesh modelsand associated regions, for example, to an electronic storage device.

FIG. 4A and FIG. 4B include exemplary methods for producingreconstructions, according to an exemplary embodiment. With slightmodifications, the method shown in FIG. 4A may serve as an exemplarymethod for producing image enhancement. FIG. 4A is a block diagram of anexemplary method 400 for producing iterative reconstruction of cardiacimages, according to one embodiment. For example, step 401A may includeinputting cardiac CT image acquisition information, for example, on anelectronic storage device. Alternately, an input may include inputtingcardiac MR image acquisition information, for example, on an electronicstorage device (step 401B). For instance, acquisition information mayinclude a set of lines in k-space acquired by one or more coils. Then,step 403A may include performing an initial cardiac CT imagereconstruction using the acquisition information (e.g., input from step401A) and any known iterative CT reconstruction technique. Analogousstep 403B may pertain to an input of cardiac MR image acquisitioninformation (rather than cardiac CT image acquisition information),where step 403B may include performing an initial cardiac MR imagereconstruction using the acquisition information (e.g., from step 401B)and any known parallel/partial MR reconstruction technique. In oneembodiment, step 405 may include inputting a set of coronary arterymodels and associated image regions from the training phase (e.g., on anelectronic storage device).

Following step 405, step 407 may include localizing the coronary arteryvessel tree centerlines within the image reconstruction, for instance,using any technique known to one of ordinary skill in the art. Step 409may then include matching each coronary artery vessel tree centerlinepoint found in the image to zero or more coronary artery vessel treecenterline points in the collection of coronary artery models input fromstep 403. The matching may be performed using any graph matchingtechnique to compute metric(s) that may describe similarity between thecoronary artery vessel tree centerline point and each point in thecollection. Exemplary metrics include spectral correspondence, minimumedit distance, etc. In one case, spectral correspondence may include aspectral method for finding consistent, geometric matches orcorrespondence between two sets of features (e.g., meshes, shapes,numbers, points, vertices, etc.). Minimum edit distance may include thelowest count of operations that would change one point to another,specifically, the coronary artery vessel tree centerline point to eachpoint in the collection. In one case, step 407 may further includedetermining a threshold value for the metric(s) that describe thesimilarity. In doing so, a collection of matched points may be created,where the matched points may contain zero or more matched points.

In one embodiment, step 411 may include determining a local image priorfor each centerline point. In other words, each centerline point mayhave an image prior that is local to that particular point. Local imagepriors may be image priors that include particular anatomical objects ofinterest. In one embodiment, the local image prior may be determined bymerging image regions associated with the zero or more matched points inthe collection of matched points. If no matched points exist for acenterline point, the point may have no associated local image prior.

In one embodiment, merging may be achieved via several methods. In oneinstance, merging may entail averaging associated image regions. Anothermethod of merging may include performing weighted averaging ofassociated image regions. For example, weights may be determined by thesimilarity metric of the associated points or the predetermined imagequality of the image, from which the associated image region wasoriginally drawn. An additional method of merging may include choosingan associated image region with greatest similarity to an image regionlocal to the centerline point in the current image reconstruction. Yetanother method of merging may include a sparse linear combination of theassociated image regions that best match the image region local to thecenterline point in the current image reconstruction.

Next, step 413 may include performing an image reconstruction using theacquisition information and image priors. For example, step 413 mayinclude blending image priors within a current reconstruction. In onecase, such blending may include applying an alpha compositing betweenthe priors and the reconstructed image. In another instance, step 413may include, for optimization-based iteration reconstruction methods,adding an extra term into the optimization that may penalize differencesbetween the reconstructed image and local priors. Step 415 may includedetermining convergence of the iterative process of steps 407-413. Forexample, step 415 may include measuring the difference between areconstructed image during two successive iterations (e.g., by computinga mean squared difference between the intensity values at all voxels)and converging if the difference is below a predetermined threshold.Then, step 417 may include outputting a converged image reconstruction,for example, to an electronic storage device and/or display. In oneembodiment, steps 403A and/or 403B through step 415 may be performedusing a computational device.

Image enhancement of a cardiac CT image may be similar in certainrespects to method 400, except that in some cases the initial stepincludes inputting a cardiac CT image, rather than cardiac CT imageacquisition information. In one instance, the cardiac CT image may beinput on an electronic storage device. As previously discussed, thedistinction between the input for enhancement versus reconstruction maybe because an image is already available to be enhanced. In addition,step 403A may be unnecessary for image enhancement, since an imageand/or reconstruction may already be available. Again, image enhancementmay not necessarily include performing an initial image reconstructionbecause image enhancement inherently already includes an availableimage. In other words, production of image enhancement may includeinputting or receiving a cardiac CT image on an electronic storagedevice and then inputting a set of coronary artery models and associatedimage regions from the training phase (e.g., on an electronic storagedevice), similar to step 403.

Next, steps analogous to steps 407-415 may be repeated untilconvergence, with the exception that the steps are performed on theinput cardiac CT image, rather than an image reconstruction (e.g., fromstep 403A). For example, a step similar to step 407 for imageenhancement may include localizing the coronary artery vessel treecenterlines within the input cardiac CT image using any known technique.An enhancement step similar to step 409 may include matching zero ormore coronary artery vessel tree centerline points from the collectionof coronary artery models, to each coronary artery vessel treecenterline point found in the image input for enhancement. A metric maythen be computed to describe similarity between each coronary arteryvessel tree centerline point in the image and each point in thecollection of models. Such a computation may be performed using anyknown graph matching technique. Example metrics include spectralcorrespondence, minimum edit distance, etc. In one embodiment, athreshold for the similarity metric may be determined. Then, acollection of matched points may be created based on the similaritymetric, where the collection of matched points may contain zero or morematched points.

Merging, similar to step 411 (e.g., to determine a local image priors)may be done using the current input cardiac CT image and not the imagereconstruction. For example, determining local image priors for eachcenterline point in an image enhancement process may include mergingimage regions associated with zero or more matched points. If zeromatched points exist for a centerline point, that point may have noassociated local prior, at least based on the input CT image and inputset of coronary artery models. Methods for merging include: averagingthe associated image regions, performing a weighted averaging of theassociated image regions (in which weights are determined by thesimilarity metric of the associated points and/or predetermined imagequality of the image (e.g., the input cardiac CT image) from which theassociated image region was originally drawn), choosing an associatedimage region with greatest similarity to an image region local to thecenterline point in the current image (e.g., input or merged image), asparse linear combination of the associated image regions to match imageregion local to the centerline point in the current image, etc.

Performing image enhancement (as analogous to step 413) may includeusing image information and image priors, for example, blending theimage priors in the current image (e.g., by applying an alphacompositing between the priors and the image). For optimization-basedimage enhancement methods, an extra term may be added into theoptimization that penalizes the difference between the image and localpriors. In one embodiment, convergence of the iterative process may bedetermined by measuring the difference between the enhanced image duringtwo successive iterations (e.g., by computing a mean squared differencebetween intensity values at all voxels) and converging if the differenceis below a predetermined threshold. Then, the method may includeoutputting a converged enhanced image (e.g., to an electronic storagedevice and/or display).

FIG. 4B is a block diagram of an exemplary method 420 for producingiterative reconstruction of abdominal images, according to oneembodiment. For example, step 421 may include inputting abdominal CTimage acquisition information, for example, on an electronic storagedevice. Step 423 may include inputting a set of organ models andassociated image regions from the training phase (e.g., on an electronicstorage device). Then, an initial abdominal CT image reconstruction maybe performed, for instance, using acquisition information and any knowniterative CT reconstruction technique (step 425).

Once such information has been acquired, steps 427-433 may be repeateduntil convergence. In one embodiment, step 427 may include localizingorgans within the image reconstruction. This step may be performed usingany known technique. Next, step 429 may include matching each organsurface mesh point found in the image to zero or more organ surface meshpoints in the collection of organ mesh models. The matching may beperformed using any graph matching technique to compute a metricdescribing similarity between the organ surface mesh point and eachpoint in the collection. As previously described, example metricsinclude spectral correspondence, minimum edit distance, etc. Step 429may further include determining a threshold of the similarity metric sothat a collection of matched points is created, where the collection ofmatched points may contain zero or more matched points. Step 431 mayinclude determining a local image prior for each surface mesh point, forinstance, by merging the image regions associated with the zero or morematched points. If a surface mesh point corresponds to zero matchedpoints, step 431 may include determining that the mesh point may have noassociated local prior. Methods of merging may include those discussedpreviously, such as, for example, averaging associated image regions,determining a weighted averaging of associated image regions, where theweights are based on the similarity metric of associated points or thepredetermined image quality of the image that provided the associatedimage region, choosing an associated image region with the greatestsimilarity to the image region local to the organ surface mesh in thecurrent image reconstruction, and/or a sparse linear combination of theassociated image regions to best match the image region local to thesurface mesh point in the current image reconstruction. Step 433 mayinclude performing an image reconstruction using the acquisitioninformation and image priors (e.g., by blending the image priors withthe current reconstruction, for instance, by applying an alphacompositing between the priors and the reconstructed image and/or foroptimization-based iteration reconstruction methods, by adding an extraterm into the optimization that penalizes the difference between thereconstructed image and the local priors). Step 435 may includedetermining convergence of the iterative process. For example, step 435may include measuring the difference between the reconstructed imageduring two successive iterations (e.g., by computing a mean squareddifference between intensity values at all voxels) and converging if thedifference is below a predetermined threshold. Step 437 may includeoutputting the converged image reconstruction, for example, to anelectronic storage device and/or display. In one embodiment, steps427-433 may be repeated until convergence and steps 425-433 may beperformed using a computational device.

The methods described in preparing sets of image regions associated withanatomical subdivisions to produce image reconstructions and/orenhancements may be applied to various forms of medical imaging. In oneembodiment, the methods may comprise a training phase and a productionphase. The training phase may include creating a set of associationsbetween image regions and anatomical subdivisions, which form an“expected” set of information against which patient-specific informationmay be assessed. The production phase may include producing imagereconstructions and/or enhancements based on the associations providedby the training phase.

Other embodiments of the invention will be apparent to those skilled inthe art from consideration of the specification and practice of theinvention disclosed herein. It is intended that the specification andexamples be considered as exemplary only, with a true scope and spiritof the invention being indicated by the following claims.

What is claimed is:
 1. A computer-implemented method of medical imagereconstruction, the method comprising: receiving an image acquisition ofa target anatomy of a patient; determining a vessel centerline modelusing the image acquisition of the target anatomy of the patient;receiving a plurality of prior vessel centerline models corresponding tothe target anatomy, the plurality of prior vessel centerline modelsbeing associated with prior image acquisitions of the patient and/orimage acquisitions of prior patients; determining a plurality ofcorresponding centerline points between points of the vessel centerlinemodel and points of the plurality of prior vessel centerline models;determining, for at least one point of the image acquisition, at leastone local image prior by matching the at least one point of the imageacquisition with a corresponding centerline point in at least one of theplurality of prior vessel centerline models, the at least one localimage prior comprising an image of a localized region of one or more ofthe prior image acquisitions of the patient and/or a localized region ofone or more of the image acquisitions of prior patients; and determiningan image reconstruction of the image acquisition of the target anatomyof the patient based on the at least one local image prior.
 2. Themethod of claim 1, wherein the plurality of prior vessel centerlinemodels are determined in a machine learning training phase.
 3. Themethod of claim 1, wherein determining at least one local image priorfurther comprises: merging image regions associated with each of theplurality of corresponding centerline points.
 4. The method of claim 3,wherein merging image regions comprises averaging image regionsassociated with each of the plurality of corresponding centerlinepoints.
 5. The method of claim 1, wherein determining an imagereconstruction of the image acquisition further comprises: blending theat least one local image prior with a corresponding portion of the imageacquisition.
 6. The method of claim 1, further comprising: iterativelyperforming steps of determining at least one local image prior anddetermining an image reconstruction until a convergence between theimage reconstruction and the image reconstruction of a prior iterationreaches a predetermined threshold.
 7. The method of claim 1, wherein atleast a portion of the image reconstruction is outputted to a storagedevice and/or display.
 8. A system for image reconstruction, the systemcomprising: at least one memory storing instructions; and at least oneprocessor executing the instructions to perform operations comprising:receiving an image acquisition of a target anatomy of a patient;determining a vessel centerline model using the image acquisition of thetarget anatomy of the patient; receiving a plurality of prior vesselcenterline models corresponding to the target anatomy, the plurality ofprior vessel centerline models being associated with prior imageacquisitions of the patient and/or image acquisitions of prior patients;determining a plurality of corresponding centerline points betweenpoints of the vessel centerline model and points of the plurality ofprior vessel centerline models; determining, for at least one point ofthe image acquisition, at least one local image prior by matching the atleast one point of the image acquisition with a corresponding centerlinepoint in at least one of the plurality of prior vessel centerlinemodels, the at least one local image prior comprising an image of alocalized region of one or more of the prior image acquisitions of thepatient and/or a localized region of one or more of the imageacquisitions of prior patients; and determining an image reconstructionof the image acquisition of the target anatomy of the patient based onthe at least one local image prior.
 9. The system according to claim 8,wherein the plurality of prior vessel centerline models are determinedin a machine learning training phase.
 10. The system according to claim8, wherein determining at least one local image prior further comprises:merging image regions associated with each of the plurality ofcorresponding centerline points.
 11. The system according to claim 10,wherein merging image regions comprises averaging image regionsassociated with each of the plurality of corresponding centerlinepoints.
 12. The system according to claim 8, wherein determining animage reconstruction of the image acquisition further comprises:blending the at least one local image prior with a corresponding portionof the image acquisition.
 13. The system according to claim 8, furthercomprising: iteratively performing steps determining at least one localimage prior and determining an image reconstruction until a convergencebetween the image reconstruction and the image reconstruction of a prioriteration reaches a predetermined threshold.
 14. The system according toclaim 8, wherein at least a portion of the image reconstruction isoutputted to a storage device and/or display.
 15. A non-transitorycomputer readable medium for use on a computer system containingcomputer-executable programming instructions for performing a method ofmedical image reconstruction, the method comprising: receiving an imageacquisition of a target anatomy of a patient; determining a vesselcenterline model using the image acquisition of the target anatomy ofthe patient; receiving a plurality of prior vessel centerline modelscorresponding to the target anatomy, the plurality of prior vesselcenterline models being associated with prior image acquisitions of thepatient and/or image acquisitions of prior patients; determining aplurality of corresponding centerline points between points of thevessel centerline model and points of the plurality of prior vesselcenterline models; determining, for at least one point of the imageacquisition, at least one local image prior by matching the at least onepoint of the image acquisition with a corresponding centerline point inat least one of the plurality of prior vessel centerline models, the atleast one local image prior comprising an image of a localized region ofone or more of the prior image acquisitions of the patient and/or alocalized region of one or more of the image acquisitions of priorpatients; and determining an image reconstruction of the imageacquisition of the target anatomy of the patient based on the at leastone local image prior.
 16. The computer-executable programming of claim15, wherein the plurality of prior vessel centerline models aredetermined in a machine learning training phase.
 17. Thecomputer-executable programming of claim 15, wherein determining atleast one local image prior further comprises: merging image regionsassociated with each of the plurality of corresponding centerlinepoints.
 18. The computer-executable programming of claim 17, whereinmerging image regions comprises averaging image regions associated witheach of the plurality of corresponding centerline points.
 19. Thecomputer-executable programming of claim 15, wherein determining animage reconstruction of the image acquisition further comprises:blending the at least one local image prior with a corresponding portionof the image acquisition.
 20. The computer-executable programming ofclaim 15, further comprising: iteratively performing steps determiningat least one local image prior and determining an image reconstructionuntil a convergence between the image reconstruction and the imagereconstruction of a prior iteration reaches a predetermined threshold.